Repository navigation
Register pydrive2:// with fsspec - #406
aryankalantri wants to merge 7 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The offline registration test raises AttributeError instead of exercising discovery, and the documentation example passes an unsupported positional argument that raises TypeError.
2 open issues
What changed in this PR
This PR addresses #354 by registering PyDrive2’s existing filesystem with fsspec, enabling discovery through the gdrive protocol without importing PyDrive2 directly.
Changes:
- Adds an fsspec entry point and declares the filesystem’s
gdriveprotocol. - Defaults the constructor’s
pathto"root"so fsspec can instantiate it without a positional path. - Adds usage documentation and offline tests for registration and missing-credential behavior.
| File | Description |
|---|---|
pyproject.toml |
Registers the gdrive fsspec entry point. |
pydrive2/test/test_fs_registration.py |
Adds offline checks for discovery, protocol declaration, and credential errors. |
pydrive2/fs/spec.py |
Declares the protocol and defaults path to "root". |
docs/fsspec.rst |
Documents fsspec factory and URL usage, including credential requirements. |
Review effort: Balanced
shcheklein
left a comment
There was a problem hiding this comment.
I reproduced both inline findings and pushed fixes in 4615fa3. The original package metadata also passed its own test while fsspec continued selecting a different backend. The new test starts a fresh Python process and resolves PyDrive2 through the public fsspec API.
I used the same entry-point group as Hugging Face and the same URL-to-storage-options hook as sshfs. PyDrive2 has its own scheme, so installing it preserves fsspec's existing Google Drive provider. DVC constructs the class directly and can continue using its own gdrive:// remotes. Hugging Face registration · sshfs URL handling · DVC adapter
One binary file-ID read took one mocked media GET. Reading two sibling files reused the shared-drive and parent-folder lookups. In a local Python 3.11 benchmark, consuming one 8 MiB in-memory chunk in 8 KiB reads improved from 192 ms to 0.69 ms. Drive network time is outside that measurement.
The backend requires credentials and exposes sequential reads. Each filesystem is rooted at one ID, so batch sibling paths under the same folder. Drive allows duplicate names and a write creates a new item. Those semantics are documented for downstream callers.
|
/test-drive |
1 similar comment
|
/test-drive |
|
/test-drive |
Restore the existing stream reader, download chunking, and buffered writer. Small-read copying and write buffering are separate concerns from registering a filesystem protocol.
|
/test-drive |


Fixes #354.
Installing
PyDrive2[fsspec]registerspydrive2://through the standardfsspec.specsentry-point group. Applications can openpydrive2://FILE_IDorpydrive2://FOLDER_ID/path/to/filewith their existing credentials. The filesystem factory defaults to My Drive and accepts a folder ID through thepathkeyword.The dedicated scheme preserves fsspec's existing
gdriveprovider. DVC'sgdrive://remotes continue to construct the PyDrive2 filesystem directly. fsspec registry · DVC adapterTests cover discovery in a fresh interpreter, case-sensitive file IDs, and sibling URLs sharing a filesystem and folder lookup. The live test writes a named file through fsspec and reads it back by ID.